Endoscopes and other medical scopes often use fluorescing agents or autoflorescence to better examine tissue. A fluorescing agent such as a dye may be injected or otherwise administered to tissue, and then an excitation light is directed toward the tissue. When the excitation light is removed, the fluorescing agent fluoresces (emits light typically at a higher wavelength than the excitation light), allowing a sensor to detect the light, which is often not in a wavelength visible to the human eye. The detected light is formatted to images, and examining the images can indicate the concentration of fluorescing agent in the observed tissue. Further, a phenomenon known as autoflorescence may occur in which tissue fluoresces light under certain conditions without a fluorescing agent. Such light can be detected as well. Images based on detected fluoresced light, known as “fluorescence imaging” (FI), are therefore useful in medical diagnosis, testing, and many scientific fields.
Other medical sensing schemes such as ultrasonic or optical coherence tomography also produce data represented to the user as images. It is often necessary to display visual color images of along with the FI or other sensor images in order to properly distinguish features and determine all desired characteristics of the tissue being investigated. The visual color images are produced by emitting light toward the tissue, and with a camera, or image sensor, taking pictures of the reflected light. Both the reflected light images and FI images can be put into image streams to show a video of the two images to the user such as a doctor using a FI endoscope.
Systems are also known which combine or overlay FI images with reflected light images of the same area to help users interpret the data in both images, such as to identify cancerous tissue. For example, U.S. Pat. No. 9,055,862 to Watanabe et al. discloses a fluorescence imaging processing device that combines a FI image with a return-light image, and processes the images with various exponential functions based on distance. Another document, U.S. Publication No. 2011/0164127 by Stehle et al. describes a method for showing endoscope images with fluorescent light. In this case, the fluoresced light is at visible wavelengths in the RGB color space and is detected with a visible light camera. The method seeks to enhance the fluoresced light portion of the image non-linearly by processing it to enhance the variations in the fluorescent light while de-enhancing the variations in other parts of the image's RGB color space. Another method for combining FI and reflected light images is found in U.S. Pat. No. 8,706,184. In this method, the visible light image is “desaturated”, that is the colors are changed to be less colorful, and in some cases the colors are completely desaturated into grey scale images. The FI image is superimposed with the desaturated image so that fluorescent features may be clearly seen relative to the more grey version of the reflected light image. All of these techniques, and others like them, suffer from distortion of colors in the reflected light image and difficulty in distinguishing FI image features when combined with the reflected light image.
What is needed are improved ways to process and display fluoresced light-based images or other medical images with visible color images. What is further needed are systems that can process a stream of images from both secondary sensors and reflected light and combine them in a manner that improves lag time in processing the image streams through the image processing device.